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Technical  Report  N#  123 


ABSTRACT 


Finite  Markov  processes  are  considered,  with  bl -dimensional  state 
space,  such  that  transitions  from  state  (n,1)  to  state  (m,j)  are  possible 
only  If  m<n+l.  The  analysis  leads  to  efficient  computational  algorithms, 
to  determine  the  stationary  probability  distribution,  and  moments  of  first 
passage  times. 


2. 


1.  INTRODUCTION 


Me  consider  Irreducible,  bl -dimensional  Markov  processes  * 

{(X(t),Y(t))f  t  >  01,  on  a  finite  state  space  ((n,1),  0  <  n  <  N,  1  <  1  <  Kn), 
for  which  transitions  from  state  (n,1)  to  state  (m,j)  are  possible  only  If 
m<n+l.  The  Infinitesimal  generator  for  such  a  process  has  a  lower  block- 
Hessenberg  form  : 
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The  diagonal  blocks  Qn  R  are  square  matrices  of  order  Kn,  0  <  n  <  N. 
Their  diagonal  elements  are  strictly  negative,  all  other  elements  are  non¬ 
negative.  The  blocks  Qn  n  •  n  ♦  "»  are  rectangular  with  appropriate  dimen¬ 
sions;  their  entries  are  non-negative.  The  rowsums  of  Q  are  equal  to  zero. 

Me  denote  by  level  n  the  set  {(n,1),  1  <  1  <  Kn>  of  states  corres¬ 
ponding  to  the  common  value  n  for  the  first  Index.  The  structure  (1.1)  of 
Q  permits  the  Markov  process  to  move  up  by  only  one  level  at  a  time,  and  to 
move  down  by  several  levels  at  a  time.  The  process  Is  said  to  be  skip-free 
to  the  right. 
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Many  models  In  queueing  theory  lead  to  Markov  processes  (or  Markov 
chains)  for  which  the  Infinitesimal  generator  (respectively  the  transition 
probability  matrix)  Is  a  (possibly  Infinite) block-Hessenberg  matrix. 

The  paradigm  for  the  lower-Hessenberg  case  Is  the  GI/M/1  queue,  for  the 
upper-Hessenberg  case.  It  Is  the  M/G/l  queue. 

Neuts  [7,9]  has  studied  In  great  detail  Markov  processes  and  Markov 
chains  with  Infinite  block-Hessenberg  matrices  which  exhibit  a  repetitive 
structure  :  for  some  n*, 

Si-V  • 

%,mm  Vl.mH  *  for  a11  »  >«•.■<»*. 

If  the  matrix  Is  lower-Hessenberg,  and  a  similar  condition  If  the  matrix  Is 
upper-Hessenberg . 

In  particular,  Neuts  has  shown  that  If  a  process  with  such  a  lower 
block-Hessenberg  matrix  Is  positive  recurrent.  Its  stationary  probability 
vector  has  a  modified  matrix-geometric  form.  For  processes  with  upper 
block-Hessenberg  matrix,  the  quantities  which  may  be  most  elegantly  studied 
are  first  passage  times  to  lower  levels;  these  first  passage  times  correspond 
to  the  busy  period  In  queueing  theory. 

There  do  not  exist  many  published  results  for  processes  with  finite 
block-Hessenberg  matrices.  Most  of  them  deal  with  block-Jacobl  matrices  (the 
blocks  Qn  m  are  zero  If  either  m  >  n+I  or  m  <  n-1).  Torrez  [12]  proposes  to 
determine  the  stationary  distribution  for  blrth-and-death  processes  In  a  random 
environment,  by  using  numerical  procedures  designed  for  band  matrices. 

Neuts  [10]  and  Carrol  et  al.  [1]  expllcltely  determine  the  stationary  probability 
distribution  for  the  PH/M/C  and  M/PH/1  queues  with  finite  waiting  room. 

Hajek  [3]  considers  a  model  with  repetitive  structure  :  Qn  n  >  j,  1  <  n  <  N-1, 
^n,n+l  *  $i  2  and  Qn+1  n  *  ^2  1*  1  **  n  ^  N"2*  Processes  with  general  block- 
Jacobl  matrices  are  analysed  In  Kellson  et  al.  [4],  and  In  Gaver  et  al.  [2], 
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where  specific  numerical  examples  are  also  given. 

Wikarski  [13]  determines,  by  purely  analytic  arguments,  the  stationary 
probability  vector  for  processes  such  that  Kq  >  Kj  >  ...  >  K^,  and  such  that 
the  matrices  Qn  ^  are  of  rank  K^j,  0  <  n  <  N-l.  We  do  not  need  these 
restrictions.  Moreover,  our  approach  yields  quantities  which  have  a  clear 
probabilistic  Interpretation. 

The  purpose  of  the  present  paper  Is  to  show  how  the  stationary  probabi¬ 
lity  distribution,  and  moments  of  first  passage  times  from  some  level  to  another 
level,  may  be  efficiently  computed. 

The  stationary  probability  distribution  Is  analysed  In  the  next  section. 
In  Section  3,  we  analyse  the  first  passage  time  from  level  n  to  a  higher  level 
m,  0  <n  <  m  <N.  The  results  in  these  two  sections  are  related  to  those  in  [2]. 

The  analysis  of  the  first  passage  time  from  level  n  to  a  lower  level  k, 

0  <  k  <  n  <  N,  Is  more  Involved,  and  Is  presented  In  Section  4.  We  shall 
observe  that  It  Is  possible  to  compute  Independently  the  stationary  probability 
distribution,  and  the  moments  of  first  passage  times  to  higher  levels.  On  the 
other  hand.  It  Is  necessary  to  compute  both  the  stationary  distribution,  and 
moments  of  first  passage  times  to  higher  levels.  In  order  to  compute  the  mo¬ 
ments  of  first  passage  times  to  lower  levels. 

In  Section  6,  we  give  some  numerical  examples  of  common-cause  of 
failure  models.  We  consider  systems  with  N  machines  and  R  repairmen.  If  n 
denotes  the  number  of  machines  In  working  condition,  the  repair  rate  is 
um1n(R,  N-n).  The  failures  occur  in  this  way  :  In  an  Interval  (t,  t+dt),  there 
Is  a  jump  from  n  to  n-l  with  probability  \n  dt  ♦  o(dt)  (Independent  failures) 
or  to  n-k,  1  <k  <  n,  with  probability  v(£)qlc(l-q)n"lcdt  ♦  o(dt)  (common-cause 
failures). 

The  results  to  follow  are  applicable,  mutatls  mutandis,  to  Markov  pro¬ 
cesses  with  upper  block-Hessenberg  Infinitesimal  generator,  and  to  finite 
Markov  chains  with  block-Hessenberg  transition  probability  matrix,  as  we  briefly 
mention  In  Section  5. 


Notations!  convention s . 


We  define  a  number  of  vectors  and  matrices  with  Indices.  Occasionally, 
we  refer  to  specific  elements  of  those.  In  order  to  somewhat  simplify  the 
notations,  we  use  Qn  n,(1,j)  to  represent  the  (1,j)th  element  of  the  matrix 

n  f  n 

Q  § . 
n,n 

We  systematically  use  “n"  to  denote  a  given  level,  “m"  to  denote  a 
higher  level,  and  "k"  to  denote  a  lower  level. 

2.  THE  STATIONARY  PROBABILITY  DISTRIBUTION 

We  denote  by  ^  the  vector  of  stationary  probabilities  associated  with 
the  Markov  process  Q  : 

RQ  -  0  .  ££  *  1,  (2.1) 

where  e  is  a  column  vector  with  unit  elements.  We  partition  the  vector  j>  as 
£  s  (l?0»  £l»  •••»  Eft)*  where  the  subvectors  have  Kn  elements  and  correspond 
to  the  states  of  level  n,  0  <  n  <  N. 

In  order  to  determine  the  vector  £,  we  need  three  preliminary  lemmas. 

Lemma  1 

Consider  a  Markov  process  on  the  state  space  {1,2,. . . ,r,r+l,r+2,. . . ,r+s}, 
% 

with  Infinitesimal  generator  Q  : 

Q  - 

where  A  Is  a  square  matrix  of  order  r,  A  is  a  rectangular  r  by  s  matrix. 

The  states  r+1  to  r+s  are  all  absorbing. 

a)  The  states  1  to  r  are  all  transient  if  and  only  If  the  matrix  A  Is  non¬ 
singular,  In  which  case  (-A-*)  >0. 


6. 


b)  The  (1,j)th  entry  of  (-A-1),  for  1  <  1,j  <  r,  Is  the  expected  amount 

of  time  spent  In  the  transient  state  j.  starting  from  the  transient  state  1, 
before  absorption  In  any  of  the  absorbing  states. 

c)  The  (l.k)th  entry  of  (-A-1  A),  for  1  <  1  <  r,  r+1  <  k  <  r+s,  Is  the  probabi¬ 
lity  that,  starting  from  the  transient  state  1,  absorption  occurs  In  the 
state  k. 

d)  The  (1,k)th  entry  of  (-A"1)(-A-1  A),  for  1  <  1  <  r,  r+1  <  k  <  r+s.  Is  the 
expectation  of  the  time  spent  In  the  transient  states,  starting  from  the 
transient  state  1,  before  absorption,  measured  on  those  paths  that  lead  to 
absorption  In  the  state  k. 

Proof.  These  results  are  well  known,  although  we  are  not  aware  of  publications 
where  they  are  proved  under  the  stated  form.  The  first  statement  Is  proved  in 
Neuts  [9],  Lemma  2.2.1;  the  second  statement  Is  a  consequence  of  Neuts  and 
Meier  [11],  Corollary  2;  the  third  statement  Is  proved  In  [2],  Lemma  1. 

To  prove  the  fourth  statement,  we  must  show  that  the  (1,k)th  entry  of  (-A  1)« 
(-A-1  A)  Is  equal  to  t  d  ^^(t),  where 

V1  kW  “  **  ^the  Process  9®*s  absorbed  In  k,  before  time  t  |  the 
Initial  state  Is  1]. 

Conditioning  on  the  state  of  the  process  at  time  t  >  0,  one  obtains,  for  h  >  0 
■  (1**n  *  *(,k h  ”,  » Vk'1)  *  •(»)■ 

,;,k<t>  ■  5  \j  'j.kW  *  *i.k  • 
or  v'(t)  *  A  v{t)  +  A  , 

where  v(t)  Is  the  matrix  formed  by  the  v1>({(t)  's.  We  already  know  that 
v(0)  ■  0,  and  11m  v(t)  ■  {-A“*)A,  therefore,  we  easily  obtain  that 


j 
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/“  t  d  V(t)  =  -  [t((-A_1)A  -  V(t))]“ 

♦  r0  ((-A_1)A  -  v(t))dt 
=  /o  ("A"1)  *'(*)  dt 

-  ("A"1)  Ev(t)]“  -  ( ~ A ) — 1 ( -A- 1 A ) . 

□ 

We  now  define  the  processes  §n,  0  <  n  <  N.  For  1  <  n  <  N,  $n  is 
the  restriction  of  the  original  process  Q,  observed  during  those  intervals 
of  time  spent  at  level  n,  before  the  process  Q  enters  level  n-1  for  the 
first  time.  The  state  space  of  §n  is  C(n,j),  0  <  j  <  Kn>.  Clearly,  all 
§n,  1  <  n  <  N,  are  transient  Markov  processes.  The  process  is  the  res¬ 
triction  of  the  process  Q  observed  at  the  lowest  level  0;  it  is  an  ergodic 
Markov  process.  We  denote  by  Cn  the  infinitesimal  generator  of  the  process 
Sn.  0  <n  <N. 

We  also  define  the  matrices  n.,0<k<n<Nas  follows  : 

n  i  k 

nn  *s  the  Probabi1ity  bbat>  starting  from  state  (n.i),  the  first 

state  visited  at  any  level  below  level  n  is  the  state  (k,j). 

Lemma  2 

The  matrices  Cn,  0  <  n  <  N  are  recursively  determined  as  follows  : 

£„  -  .  (2.2) 

Cn-  V»  *  W  •»  <"<"-!•  (2-3) 

Proof.  The  equation  (2.2)  Is  obvious  :  starting  from  level  N,  the  process 
§N  terminates  as  soon  as  the  process  Q  enters  any  of  the  levels  0  to  N-1. 
Meanwhile,  the  transitions  are  governed  by  QN  N. 

Consider  n  fixed,  1  <  n  <  N-1,  and  let  Zn( t ) ,  t  >  0  be  defined  as  follows. 

Zn(t)  »  1  if  the  process  5n  is  in  state  (n.i)  at  time  t. 
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Assume  that  Zn( t )  =  i  and  Zn(T  +  dx)  *  j  *  i.  Since  j  *  i,  a  transition 
must  have  occured  in  the  process  Q.  Zn(x  +  dx)  =  j  if  and  only  if  one  of 
the  following  events  have  occured. 

a.  The  transition  Is  from  (n,1)  to  (n,j),  this  happens  with  probability 
Qn,n  dT  +  °<dT>* 

b.  The  transition  is  from  (n,i)  to  (n+l,h)  for  some  h,  and  the  process  Q 
returns  to  (n,j),  after  spending  an  unspecified  amount  of  time  at  level 
n+1  or  higher,  before  visiting  any  other  state  at  level  n  or  lower.  This 
happens  with  probability  Qntn+i0.h)  <*x  n^j  n(h,j)  +  o(dx). 

It  is  now  clear  that 


which  proves  (2.3)  for  n  >  1.  The  proof  for  n  =  0  is  Identical. 
Lemma  3 


The  matrices  nn((,  0  <  Ic  <  n  <  N  are  recursively  determined  as 


follows 


Vk  "  ('CN  >  QN,k  ’  for  0  <  k  <  N-l.  (2.4) 

VK'llVUW'  for  0<k  <„<»-!  (2.5) 
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Proof.  The  equation  (2.4)  Is  an  Immediate  consequence  of  Lenina  1  and  (2.2). 
A  simple  probabilistic  argument  leads  to 


nn,k  *  ("^n,n^n,k  +  ^n,n+l  nn+l,k  +  ^n,n+l  nn+l,n  nn,k^* 


which,  together  with  (2.3),  Implies  (2.5). 


Remark.  The  matrices  0  <  n  <  N  and  n„  0  <  k  <  n  <  N,  must 

n  n,K 

be  numerically  computed  together,  as  Indicated  below. 

Algorithm  A 

A.l  CN  Qn>n; 

A. 2  for  k  from  0  to  N-l  do  determine  from  (2.4); 

A. 3  for  n  from  N-l  to  1  step  -1  do 

begin  detenu ne  Cp  from  (2.3); 

for  k  from  0  to  n-l  do  determine  n_  from  (2.5) 

“  “  '  ■  '  lifK 

end; 

A. 4  determine  CQ  from  (2.3)« 

Me  are  now  In  a  position  to  prove  our  first  main  result. 


Theorem  1. 

The  vectors  2^,  0  <  n  <  N,  are  determined  by  the  equations 


fio  ^0  * 

En-Vl  Vl,.  I-';1)- 

N 


(2-6) 

(2.7) 


(2.8) 


Proof.  From  the  structure  (1.1)  of  Q,  It  results  that  the  system  £.  Q  «  0 
may  be  decomposed  into 

N 

Bo  Q0,0  +  r.  Bv  %,0  * 

N 

fik-l  Qk-l,k  +  Bk  <>k,k  +  *  Bv  Qv,k  ■  for  1  <  k  <  H-l, 


(2.10) 


Bn-1  qn-i,n  +  Bn  ^n.n  *  -  (z-n) 

From  (2.11)  and  Lemma  2,  ^  -  -fin.!  QN-1,N  Qn!n  *  Bn-1  QN-1,N  (-QN^NJ • 
which  proves  (2.7)  for  n  *  N. 

The  remainder  of  the  proof  uses  an  induction  argument  In  two  steps. 
Induction  assumption  1  :  for  a  given  k,  1  <  k  <  N-l,  the  equation  (2.7) 
holds  for  k+1  <  n  <  N. 

In  order  to  prove  that  (2.7)  holds  for  n  *  k,  we  first  need  to  show 


v*.  Ev  Qv.k  "  fit  <’Ct)  nt,k  *  for  k+l  <  t  <  l 


(2.12) 


Clearly,  (2.12)  holds  for  t  ■  N,  by  (2.4).  An  induction  argument  will  be 
used  to  prove  (2.12)  for  arbitrary  t  >  k+1. 

Induction  assumption  2  :  for  a  given  a,  k+1  <a+l  <N,  the  equation  (2.12) 
holds  for  a+1  <  t  <  N. 

We  prove  that  It  holds  for  t  ■  a,  by  using  successively  the  assump¬ 
tion  2,  the  assumption  1,  and  (2.5). 


1  fiv  Qv,k  "  Bo  ^a.k  +  fia+1  ('Co+l>  "a+l.k 

v»a 

*  E«  W«.k  *  Qa,a.l  n.*l.k>* 

■  E.  ■ 
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Finally,  to  show  that  (2.7)  Is  satisfied  for  n  *  k,  note  that 
(2.10)  can  be  written  as 


N 

-  "  £k-l  Qk-l,k  +  £k  Qk,k  +  v*k+1  %,k 

m  fik-1  Qk-l,k  +  fik  Qk,k  +  £k+l  (-Ck-»-l)  nk+l,k 
*  2k-l  fyc-l,k  +  £fc  ^k.k  +  Qk,k+1  nk+l,kJ- 


The  equation  (2.6)  Is  proved  In  the  same  way.  Since  CQ  Is  the 
Infinitesimal  generator  of  a  finite.  Irreducible  Markov  process,  the  equation 
(2.6)  has  a  unique  solution,  up  to  a  multiplicative  constant,  and  that 
constant  Is  determined  by  (2.8).  a 

Me  have  thus  shown  that  the  matrices  CR,  0  <  n  <  N,  are  the 
crucial  quantities  needed  to  determine  the  stationary  probability  distribution. 
Me  shall  observe  In  Section  4  that  the  same  matrices,  together  with  the 
matrices  k,  0  <  k  <  n  <  N,  play  a  central  role  In  the  evaluation  of  first 
passage  times  from  a  given  level  n  to  a  lower  level  k.  We  must,  however, 
first  examine  the  first  passage  times  to  higher  levels. 


3.  FIRST  PASSAGE  TIME  TO  HIGHER  LEVELS 

Me  denote  by  xn>n,  the  first  passage  time  from  level  n  to  a  different 

level  n'  :  rn  -  inf  {t  >  0  :  X(t)  -  n'|X(0)  -  n>,  0  <  n.n*  <  N,  n  ♦  n\ 

We  are  concerned  In  this  section  with  the  first  passage  times  t  for 

n,m 

m  >  n.  We  need  to  first  establish  some  preliminary  results 

We  define  the  processes  Sfll  0<n<N1n  a  similar  fashion  to  the 
processes  Sfl.  For  0  <  n  <  N-l,  the  transient  Markov  process  $n  is  the  restric¬ 
tion  of  the  process  Q,  observed  during  those  Intervals  of  time  spent  at  level  n, 
before  the  process  Q  moves  upward  to  level  n+1  for  the  first  time.  The  ergodlc 
process  SN  is  the  restriction  of  the  process  Q  observed  at  the  highest  level  N. 
We  denote  by  Cn  the  infinitesimal  generator  of  the  process  Sn,  0  <  n  <  N. 


12. 


We  also  define  the  matrices  r.  0  <  n  <  ■  <  N  as  follows  :  r  fl.j)  Is 

f I  fM  if  •01'  1 

the  probability  that,  starting  from  state  (n,1),  the  first  state  visited  at 
level  m  Is  the  state  (m.j).  More  formally,  rn  n(1,j)  •  FtY(Tn  m>  -  j|X(0)  - 
n,  Y (0)  -  11. 

Clearly,  r  e  -  e,  for  all  n  and  m.  (3.1) 


Leona  4. 

The  matrices  Cn,  0  <  n  <  N,  are  recursively  determined  as  follows 


C0  *  90,0  • 


n-1 


C"  ■  Vn  ♦£  V 


The  proof  Is  analogous  to  that  of  Leona  2  and  Is  omitted  here. 


(3.2) 

(3.3) 


Lemma  5 


The  matrices  rn>|n,  0  <  n  <  m  <  N  are  recursively  determined  as 

follows 

rn,n+l  "  K1)  Vn+1  *  f°rO<n<N‘l. 

r  »  r  r 
n,m  n,a-l  *-l,m 

m 

■  n  rt  .  t  ,  for  0  <  n  <  N-2, 
t-n+1  z 

n+2  <  m  <  N. 


(3.4) 

(3.5) 

(3.6) 


Proof.  For  n-0,  the  equation  (3.4)  is  an  lonedlate  consequence  of  Leona  1 
and  (3.2).  For  1  <  n  <N-1,  observing  the  system  at  the  first  departure  from 
level  n  leads  to 


,-l 


n-1 


rn,n+l  “  ^n.n^n.n+l  +  kfg  rk,n  rn,n+l^’ 


which,  together  with  (3.3),  Implies  (3.4).  To  prove  (3.5),  since  Q  has  the 

structure  (1.1),  one  merely  decompose  the  probabilities  In  r  _  according  to 

n  |R 
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the  first  state  visited  at  level  m-1.  Equation  (3.6)  Is  now  obvious. 

I 

The  matrices  c  ,  0  <  n  <  N  and  r*  0  <  n  <  m  <  N,  must  be  numeri- 
n  n*m 

cally  computed  together,  as  Indicated  below. 

Algorithm  B 


determine  rQ  j  from  (3.4); 

B.2  for  m  from  1  to  N-l  do 

begin  determine  CB  from  (3.3) ; 

determl ne  rffl  from  (3.4); 

for  n  from  0  to  m-1  do  determine  rn  ^ j  from  (3.5) 

end; 

B.3  determine  CN  from  (3.3). 


We  now  define,  for  x  >  0,  0  <  n.n'  <  N,  n  ♦  n',  1  <  1  <  Kn, 


<«•  '  »•  f<°>  ■  "•  <3-7> 


we  denote  by  n , (e)  the  matrix  such  that  n«(C)(1,j)  equals  the  Laplace- 
Stleltjes  transform  of  the  possibly  defective  distribution  gn  .  ,  ,(.). 

Clearly,  since  the  Markov  process  may  only  move  up  by  one  level  at 
a  time,  we  have  that 


6n,m^  *  Sn,m-1^)  *Vl,m^ 

m 

■  n  G»  ,  +(0,  forO<n<N-2, 
t-n+1 


(3.8) 

(3.9) 


n+2  <m  <N. 
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A  simple  argument  conditioning  on  the  state  of  the  process  at  time  h  >  0 
shows  that 


9n,1;n+l,j(x+h)  ^  +  ^n.n^l.l  9n,1  ;n+l,j(x) 

+  ^^n.n^l.v  h  9n,v;n+l,j(x) 
n-1  *k  Kn 

+  lc«0  vfA^l.v  h  9k.v;n,y  *  9n,ul  n+l,j 
+  (Qn|n+i)ij  h  +  °(h).  for  1  <  n  <  N-1, 


(x) 


where  *  denotes  the  Stleltjes  convolution,  from  which  we  conclude  that 


'  Gn.«l<«  '  Vn  <W'>  *  «n.n*l 

(3.10) 

*  £  V  W'*  ♦»»<•<*■»• 

and  similarly, 

5  Gfl.lW  *  <>0.0  %,i<0  ♦  %,y  (3-H) 

This  equations  may  be  written  as 

W«  •  °n<«  V»1  •  ,or0<"<"-1'  <3  “) 

«•»>»  D0({)  ■  (el  -  Qq  o)'1  .  (3.13) 

°n<«  ■  («  t  Vk  '•'!<»<  "->•  (>•») 

Of  course,  we  have  Gn#m(0)  *  ,  and  from  (3.3)  Dn(0)  ■  (-C"1), 

0  <  n  <  m  <  N. 


15. 


Let 

un  „• 

n,n 

a 

at  -  —  ■  ■■ 

95 

6n,n,(5>  |c-0  ’ 

(3.15) 

and 

u_ 

—4i  ,n 

■  IL  .  e 
n,n  — 

,  for  0  <  n,n*  <  N,  n  *  n* . 

(3.16) 

We  have 

W’-l)  ■  «’»,.•  Y('n..  *  “>  •  ■  "•  »(»)  ■  <>• 

and  *  E[thj.  I  *<°)  *  "•  ><°>  '  '>• 


Theorem  2. 

The  expected  values  of  the  first  passage  times  to  higher  levels 
are  given  by  the  following  relations. 


“0,1  ”  ("Co1)  £. 

Vn+1  *  ^  %.k  Mfc,n>-  l<n<N-l, 


^fi.m  “  ^n,m-l  +  rn,m-l 


for  0  <  n  <  N-2, 
n+2  <  m  <  N . 


(3.17) 

(3.18) 

(3.19) 


Proof.  The  equation  (3.17)  Iwaedlately  results  from  Lemma  1.  The 
equations  (3.19)  are  directly  proved  by  using  the  strong  Harkov  property  : 

■  E  [Tn.«  !  X<°>  -  "■  Y<°>  -  11 
-  E  I  X(0)  -n,  Y(0)  -  1] 

+  EIEIVl,m  I  ^(Tn,m-1  +  °)HX(°)  "  "•  Y<°>  *  11  ’ 


Conditioning  on  the  first  level  visited  after  level  n,  and  using  Lemma  1, 
we  obtain 


16. 


-1  ,,_A  -1 
u  .  *  (-Q„  „)  e  +  I  (-Q_  -)  Q_  l.  Ur, 
hi 1  n,n  i  /.  n,n  n,K  HtifHl 


n-1 

I 

k*0 

n-1 


(‘^n,n)  -+  k^g  ('^n,n^  ^n,k  ^.n  +  rk,n  ^.n+l^' 


by  (3.19).  Hence, 


n-1  n-1 

^n,n  +  k^0  ^n,k  rk,n^  ^.n+l  +  k*Q  ^n,k  ^k,n^' 


which,  together  with  (3.3),  proves  (3.18).  a 

Remarks . 

a.  Another  argument,  using  the  equations  (3.12),  (3.15)  and  (3.16), 
requires  the  evaluation  of  the  matrix  d  Dn(c)/H  •  This  is  done  In 
Appendix  A.  Similarly,  higher  moments  of  first  passage  times  may  be 
obtained  by  purely  routine,  albeit  tedious,  manipulations. 

b.  The  vectors  u^  may  be  computed  together  with  the  matrices  rn  m,  by 
appropriately  modifying  Algorithm  B. 


4.  FIRST  PASSAGE  TIME  TO  LOWER  LEVELS 

We  denote  by  Tfl  k  the  first  passage  time  from  level  n  to  a  lower 

level  k.  In  the  preceding  section,  we  have  in  essence  decomposed  the  first 

passage  time  from  level  n  to  a  higher  level  m.  Into  a  sum  of  first  passage 

times  from  n  to  n+1,  from  n+1  to  n+2,...,  from  ra-1  to  m.  Since  It  Is 

possible  to  move  down  several  levels  in  one  transition,  no  such  simple 

decomposition  Is  possible  for  +_  L.  Rather,  we  must  expllcltely  consider 

n 

the  first  level  visited  below  n,  starting  from  level  n.  In  order  to  do  this, 
we  define  the  first  passage  time  Tn  as  follows 

Tn  •  Inf  {t  >  0  :  X(t)  <  n  |  X(0)  -  n>  , 

■  nl1n  »  0  <  k  <  n-1),  for  1  <  n  <  N. 
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We  define,  for  x  >  0,  0  <  k  <  n  <  N,  1  <  1  <  KR,  1  <  j  <  1^, 

9n,i;k,j(x)  ■  p  [Tn  <x*  x<Tn  +  °)  "  k*  ¥<Tft  +  °)  “  jlx(d)  “  ",  *(0)  ■  U. 

we  denote  by  Sfl  k(e)(1,j)  the  Laplace-Stleltjes  transform  of  the  possibly 
defective  distribution  gnj.k  j(-)»  and  Sn|k(C)  the  matr1x  w*th  entries 
given  by  fi„fk(5)(1  .j) • 

An  argument  similar  to  the  one  used  to  obtain  (3.10)  and  (3.11)  shows 

that 

5  5N,k(*>  "  <>N.k  +  <*".N  W(C)‘  for  0  <k  <N-1  (4.1) 

and  *  Sn>k(5)  «  Qn  <  .  Qn>n  ^k(C)  (4.2) 

+  <Wl  «ml.k(C)  +  5n.k^*  f or  0  <  k  <  n  <  N-l. 


These  equations  may  be  written  as 


Vk(t) 

“  QN,k  ’ 

for  0  <  k  <  N-l , 

(4.3) 

fin,k(«) 

“  Bn«)  Wn.k  + 

Qn,n+1  8n+l,ktc)J> 

for  0  <  k  <  n  <  N-l» 

(4.4) 

where  DN(c)  • 

‘  qn,n}  1  * 

(4.5) 

Dn(0  - 

W  -  Qn.n  -  Qn, 

,n+l  8n+l,n^^  ’ 

for  1  <n  <  N-l. 

(4.6) 

Obviously,  we 

have  that  5.  fc(0)  -  n„  t,  and  (Lemma  2)  B„(0)  ■  (-CI1). 
n  }K  n  n  n 

Observe  that 

nn  k  e  <e,  for  0  <  k  <  n  <  N, 
n-l 

r  n„  .  e  ■  e,  for  1  <  n  <  N. 
k«0  n,x  "  ” 


(4.7) 
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Let 


°n,k  "  '  —  5n,k(s)  ^-O* 


(4.8) 


0nik(1,j)  -  E[Tn,X(Tn  +  0)  •  k,  Y(Tn  +  0)  .  j|X(0)  -  n,  Y(0)  -  1]. 


Lemaa  6. 


The  matrices  0n  k  are  recursively  determined  as  follows 


,k  "  (“^jj1)  nN,k  *  for  0  <  k  <  N-l, 


N 


(4.9) 


^n,k  "  ^n  ^nn,k  +  ^n,n+l^n+l,k  +  ®n+l,n  nn,k^*  (4.10) 

for  0  <  k  <  n  <  N-l. 

Proof.  It  results  from  Lemma  1  that 
«“l\/_r-l 


which  together  with  (2.4)  proves  (4.9).  For  n  <N-1,  we  condition  on 
the  first  level  visited  after  level  n,  and  we  obtain  by  using  Lama  1  and 
the  strong  Harkov  property  that 

^n,k  “  nn,k  +  ^n,n  ^n,n+l^n+l,k  +  ®n+l,n  nn,k  +  nn+l,n  ®n,k^’ 


hence 


^n,n  +  ^n.n+l  nn+l,n^  ^n,k  "  ’  ^nn,k  +  ^n,n+l  ^n+l,k  +  ^n+l,n  nn,k^' 


which,  together  with  (2.3),  proves  (4.10). 


Corollary  1. 


The  vectors  u_  defined  as 

— fl 

n-l 

u_  *  r  0_  e 
-n  k-0  n,k  - 


(4.11) 


are  recursively  determined  as  follows 


W. 


h  m  ("Cm1)  i  •  (4-i2) 

%  *  (-CnXH£  +  Qn.n+l  W  *  for  1  <  "  <  «*-!•  (4-*3) 

□ 

This  corollary  Immediately  results  from  Lemma  6  and  (4.7). 

The  Laplace-Stleltjes  transforms  of  the  distribution  functions 

for  the  first  passage  times  t  are  given  by  the  matrices  G„  .(e).  If 

n  n,K 

one  decomposes  the  trajectories  from  level  n  to  level  k,  according  to  the 
first  level  v  visited  below  n,  one  Immediately  obtains 

Gn.k(5)  "  5n,k<*>  +  fin fV<«>  Gv.k<«> 

n-1  ^  ^ 

+  vik+l8n.v^>Gv,k^)»  for  0  <  k  <  n  <  N , 

from  which  the  moments  of  first  passage  times  to  specific  lower  levels 
may  be  derived.  The  first  moments  may  be  more  easily  obtained  by  using 
the  strong  Markov  property.  The  next  theorem  Is  stated  without  proof. 


Theorem  3. 


The  expected  values  of  the  first  passage  times  to  lower  levels 
are  given  by  the  following  relations 


—1.0 

^n.k 


“l- 


(♦.15) 


2.  *  £  V,  Vk  *  "n.v  Vk'  'or  0  <  k  <  n  <  «.  (4.16) 


where  the  matrices  n_  are  determined  by  Lenina  3,  and  the  vectors  u  . 

n»V 

for  v  <  k-1  are  determined  by  theorem  2.  o 


20. 


We  observe  here  a  major  difference  between  the  first  passage  times 
to  higher  and  to  lower  levels.  We  have  observed  In  the  preceding  section 
(Remark  b.  after  Theorem  2)  that  the  vectors  u^  m  for  m  >  n  may  be  computed 
In  one  loop,  starting  from  n  ■  0,  to  n  *  H-l.  The  evaluation  of  the  vectors 
Ujj  k  for  k  <  n  requires  two  steps.  One  for  computing  the  vectors  uf),  starting 
from  n  *  N,  to  n  *  1.  In  the  second  step  are  computed  the  vectors  u  ., 

»K 

starting  from  n  *  1,  to  n  *  N.  We  observe  also  that  the  vectors  u„  for 

— n  ,m 

m  >  n  have  to  be  computed,  even  If  one  Is  only  Interested  In  the  first  passage 
times  to  lower  levels. 

It  Is  clear  that  one  can  combine  the  computation  of  the  stationary 
probability  distribution,  and  the  expected  first  passage  times,  as  is  done  In 
the  algorithm  C  given  In  Appendix  B. 

It  is  much  easier  to  obtain  moments  for  the  first  passage  time  from 
level  n  to  any  level  at  or  below  k  <  n-1.  Let 

Tn,k  *  1nf  tt  >  0  :  X(t)  <k|X(0)  -  n}, 

*  min  {t_  :  0  <  v  <  k) ,  for  0  <  k  <  n  <  N. 

n  |V 

We  denote  by  Gn  k(t)  the  matrices  with  entries  equal  to  the  Laplace-Stleltjes 
transforms  of  the  distribution  functions  for  the  first  passage  times  T„  t. 
Also,  let 

3  <v 

^n,k  “  “  ®n,k^U*0  * 

* 

and  iL  l  ■  U_  L  e  ,  for  0  <  k  <  n  <  N. 

— n,x  n,x  — 


If  we  condition  on  the  first  level  v  visited  below  n,  we  obtain 


A  8»-<e) 


n-l 

+  z  5_ 

v»k+l  ^ 


»V 


(0  GVtk(0 


from  which  the  moments  of  Tn  k  may  be  obtained.  Again,  it  is  easier  to  deter¬ 
mine  the  first  moments  by  using  the  strong  Markov  property.  The  next  theorem 


L 


Is  stated  without  proof. 


Theorem  4. 


The  expected  values  of  the  first  passage  time  from  level  n  to  any 
level  at  or  below  k,  are  given  by 


U  -  3  U 

-n,n-l  — n 


and 


■X. 

U_  t 

— n,k 


h-1 
+  r 


V  for  0  <  k  <  n  <  N, 

v*k+l 


(4.17) 

(4.18) 


where  is  defined  in  (4.11).  o 

The  vectors  2f  v  may  be  computed  in  one  loop,  beginning  with 
k  *  N-l,  n  *  N,  and  progressively  decreasing  k  until  k  «  Oj  for  each  value 
of  k,  one  computes  the  vectors  2^  k  to  u^+1  In  Section  5,  we  shall  refer 
to  Algorithm  0  (in  Appendix  B),  which  is  designed  to  recursively  compute  the 
vectors  Ujj  N-l  >k  >0. 


5.  GENERAL  REMARKS 


Remark  a. 

We  firstly  comment  on  Equation  (2.7).  The  (1,j)th  entry  of  the 
matrix  Qn.1>n  (-C^1)  Is  equal  to 

1  1 

lVl,n  Vl.n(<*k)(-V)C‘.J). 


The  factor  Qn_i>nC* »*cV(“Qn_i  n_^)(1  *i )  1  s  the  probability  that, 
upon  leaving  the  state  (n-1,1),  the  process  Q  moves  to  the  state  (n,k). 

The  (k,j)th  entry  of  (-C^*)  Is  the  expected  time  spent  by  the  process  in 
state  (n,j),  starting  from  (n.k),  before  hitting  any  state  at  a  lower  level, 
by  Lenina  1  and  the  definition  of  Cn.  If  that  lower  level  Is  below  level  n-1, 
the  process  will  be  forced,  by  the  structure  (1.1)  of  Q,  to  pass  through  level 
n-1  before  reaching  again  level  n. 

Therefore,  it  results  from  (5.1)  that  [Qn-1  n(-C“*)](1,J)  Is  equal  to 
(-Qn_i  n_j)(1»1)time<.  ;he  expected  time  spent  in  the  state  (n,j),  before  the 
next  return  to  level  n-1,  given  that  the  process  Q  starts  in  the  state  (n-1,1). 
This  Is  exactly  the  Interpretation  of  Neuts'  matrix  R  ([9],  Theorem  1.7.2,  and 
[8]),  and  shows  that  (2.7)  Is  the  counterpart  to  the  matrix-geometric  solutions. 

Remark  b. 

If  the  matrix  Q  is  upper  block-Hessenberg,  Instead  of  having  the  form 
(1.1),  then  our  results  can  be  modified  In  an  obvious  way.  In  that  case,  the 
first  passage  times  to  lower  levels  may  be  directly  computed,  via  an  algorithm 
derived  from  Algorithm  B,  while  the  first  passage  times  to  higher  levels  are 
more  Involved.  Also,  the  stationary  probability  vector  Is  computed  by  determl- 
n-jng  first  the  vector  (up  to  a  multiplicative  constant)  and  then  recursively 
the  vectors  . £q* 

This  last  observation  points  to  the  reason  why  It  Is  difficult  to  eva¬ 
luate  the  stationary  probability  distribution  for  Infinite  upper  block-Hessenberg 
matrices. 

We  briefly  examine  now  the  problem  of  obtaining  a  good  approximation  for 
the  stationary  distribution  In  such  a  case.  Neuts  [7]  and  Lucantonl  and  Neuts 
[6]  consider  Infinite  upper  block-Hessenberg  matrices  of  the  following  form  : 


and  stationary  probability  vector  x  =  (Xg,  Xj,  x^,  . ..).  An  explicit  form 
is  found  in  [6,7]  for  Xq  and  x^,  as  well  as  complicated  but  tractable  expres¬ 
sions  for 

m,  =  z  \>  x  e  , 

1  n  —o  — 
v=0 

*  2 

m2  =  Z  v  ^  e . 
v=0 

From  these,  a  heuristic  method  is  proposed  to  truncate  the  matrix  Q  at  some 
level  N,  and  it  is  suggested  that  x^,  x3,  ....  be  determined  by  a  block 
Gauss-Seidel  iterative  procedure. 

An  alternative  method  would  be  to  determine,  as  indicated  in  Section 
1,  the  vector  £  corresponding  to  the  finite  matrix,  and  to  compare  the  vectors 

t0  (^0*— ♦  in  order  check  whether  the  level  N  is  properly  ch<>:..»n 

% 

Also,  we  may  truncate  the  matrix  Q  on  the  basis  of  first  passage  times 
argument.  Instead  of  using  m^  and  m2<  It  is  very  easy  to  adapt  Algorithm  D 
(In  Appendix  B),  in  order  to  compute  successively  the  expected  first  passage 
time  from  level  0  to  any  level  at  or  above  m,  for  m  ■  1,2,...  .  One  may  then 
choose  N  to  be  the  smallest  value  of  m  for  which  the  expected  first  passage 
time  exceeds  a  given,  large  value.  When  N  is  determined,  the  vector  £  can  be 
readily  obtained,  since  most  of  the  preliminary  computations  are  already 
performed. 
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Remark  c. 

We  have  not  considered  first  passage  times  to  specific  states,  but 
have  concentrated  our  attention  on  first  passage  times  to  levels.  The  reason 
is  that,  in  a  number  of  applications,  such  as  chains  in  a  random  environment, 
the  levels  themselves  are  of  importance.  Nevertheless,  it  is  not  very  diffi¬ 
cult  to  extend  our  analysis  to  the  study  of  x_  *  inf{t  >  0  :  X(t)  =  m, 

Y(t)  =  j  |  X(0)  =  n,  Y(0)  =1}.  With  the  results  of  Sections  3  and  4,  we  can 
observe  the  process  between  successive  visits  to  level  m,  until,  for  the  first 
time,  the  state  (m,j)  is  visited  before  the  process  leaves  level  m. 

Remark  d. 

We  have  only  considered  continuous -parameter  Markov  processes.  Our 
analysis  may  be  immediately  adapted  to  discrete-time  Markov  chains.  The  major 
difference  is  that,  in  all  our  equations,  the  matrices  (-C*1)  and  (-C”1)  will 
be  replaced  by  fundamental  matrices  for  absorbing  Markov  chains  (Kemeny  and 
Snell  [5],  Section  3.2). 

6.  NUMERICAL  EXAMPLES 


We  have  computed  some  characteristics  for  different  machine-repairmen¬ 
like  processes,  using  the  concepts  of  common-cause  failure  (described  in  the 
Introduction)  and  of  randomly  varying  environments.  We  have  considered  four  dif¬ 
ferent  types  of  systems,  described  below  In  Increasing  order  of  their  complexity. 

We  have  computed,  for  each  system,  the  stationary  distribution  £  and  the 
vectors  ir  L.  To  do  this,  we  have  used  Algorithm  E  (In  Appendix  B),  which  Inclu- 
des  parts  of  both  Algorithms  C  and  D. 


Me  shall  not  attempt  to  answer  any  specific  question  about  such 
systems.  Rather,  we  shall  describe  the  qualitative  and  quantitative  diffe¬ 
rences  In  their  behaviour.  We  shall  use  here,  as  the  main  performance  measure, 
the  expected  times  tfN  k  until  there  remain  k  or  less  operating  machines,  start¬ 
ing  from  a  fully  operational  system. 

Type  I.  The  first  model  Is  the  classical  machine- repairmen  process.  We  con¬ 
sider  a  system  of  N  machines  and  R  repairmen,  with  failure  and  repair  rates 
respectively  given  by  x*  and  y.  The  Infinitesimal  generator  is  in  this  case 
a  trl -diagonal  matrix. 

Type  II.  This  Is  a  simple  common-cause  failure  model.  In  superposition  to 
a  machine-repairmen  system,  with  N  machines,  R  repairmen,  failure  and  repair 
rates  given  by  x  and  u.  we  consider  an  Independent  point  process  with  exponen¬ 
tial  Interevent  time  distribution,  with  parameter  v  :  the  process  of  common- 
cause  failures.  Whenever  a  cannon  cause  failure  occurs,  each  operating  machine 
has  a  probability  q  of  falling,  independently  of  the  other  machines.  Thus,  if 
there  are  n  operating  machines,  the  number  of  machines  that  fall  due  to  common- 
cause  has  a  binomial  distribution,  with  parameters  n  and  q.  The  Infinitesimal 
generator  has  the  form  (1.1),  with  blocks  of  order  1. 

We  define  as  system  II. A  the  camon-cause  failure  system  with  N  =  10, 
R  ■  1,  x  «  .005,  u  *  1,  v  «  .01,  q  *  .2.  If  1/y  Is  equal  to  one  hour,  then 
1/x  «8  days,  and  1/v  «4  days.  The  system  I. A  Is  a  machine-repairmen  system 
with  N  ■  10,  R  ■  1,  x'  ■  .007,  y  «  1.  That  value  for  x1  Is  chosen  so  that  both 
systems  are  globally  similar  In  the  following  sense  :  when  n  machines  are  ope¬ 
rating,  they  both  have  the  same  expected  number  of  failures  in  an  Interval  of 
length  df  :  this  Is  n(x  dt  +  q  v  dt)  +  o(dt)  »  .007  n  dt  for  system  II. A,  and 
n  x'  dt  +  o(dt)  ■  .007  n  dt  for  system  I. A. 


We  give  In  Table  I  the  stationary  probability  distribution  for  these 
two  systems.  We  observe  that  both  systems  have  high,  nearly  equal,  probabili¬ 
ties  of  having  most  machines  In  operation.  Nevertheless,  the  two  systems  are 
different  since  the  distribution  II. A  has  a  longer  tall  towards  the  states 
with  few  operating  machines. 

This  difference  Is  strikingly  apparent  on  Table  II,  where  we  give  the 
expected  times  k,  0  <  k  <  9.  For  Instance,  If  we  compare  u1Q  5  for  both 
systems,  we  observe  that  the  average  time  until  5  machines  or  more  are  under 

3 

repair.  Is  smaller  by  a  factor  10  for  the  common-cause  failure  system. 

We  conclude  that  multiple  failures  have  a  profound  effect  on  the  dyna¬ 
mic  behaviour  of  the  system.  In  support  of  our  conclusion  that  multiple  failures 
are  the  main  cause  for  difference,  we  observe  that  both  systems  have  Identical 
repair  facilities  (R  »  1,  u  ■  1),  and  that  the  stationary  expected  number  of 
failures  per  unit  of  time  (henceforth  denote  by  <p)  are  nearly  equal  :  they  are 
respectively  given  by 

10 

<pT  ,  ■  £  n  p  i'  ■  .0694815, 
n«0  n 

10 

and  <pjj  ^  ■  £  n  pn(X  +  vq)  *  .0693535. 
n«0 

Type  III.  We  assume  now  that  the  failure  and  repair  parameters  change  according 
to  an  Independent  environmental  process  :  they  are  respectively  given  by  Xj,  Vy, 
vj  and  qj  when  the  environment  state  Is  j,  1  <  J  <  K.  We  assume  that  the  envi¬ 
ronmental  process  Is  an  Irreducible  Markov  process,  with  Infinitesimal  generator 
T.  The  system  Is  now  described  by  a  pair  of  Indices  (n,j),  where  n  denotes  the 
number  of  operating  machines,  and  j  denotes  the  state  of  the  environment.  The 
Infinitesimal  generator  has  the  form  (1.1),  with  blocks  of  order  K.  A  typical 
example,  with  N  ■  4,  R  ■  2  and  K  •  3  Is  displayed  In  Appendix  C.  Note  that  the 


off-diagonal  blocks  are  diagonal  matrices  because  the  only  possible  transi¬ 
tions  are  from  (n,j)  to  (n,j*),  with  j  ♦  j',  or  (n,j)  to  (n',j),  with  n  *  n'. 

We  firstly  define  a  system  with  two  environments.  In  environment  1 
(the  off -environment) ,  no  common-cause  failure  may  occur.  In  environment  2 
(the  on-environment),  common-cause  failure  may  occur  at  the  rate  v*.  Indivi¬ 
dual  failures  occur  with  the  same  rate  x  In  both  environments. 

In  order  to  make  meaningful  comparisons  with  the  system  II. A,  we  have 
chosen  v*  so  that  the  expected  time  between  two  common-cause  failures  Is  equal 
to  1/v  *  (.Olf1,  the  same  as  for  the  system  II. A.  The  time  between  two  succes 
slve  common-cause  failure  has  a  PH-dlstrlbutlon  (Neuts  [9],  chapter  2),  with 
representation  (8,  V),  where 

-<T21+  T21 

a  ■  [l  o],  v  - 

T12  ~T12  . 

and  Its  first  moment  is  given  by  -8  V"1  e  *  (T21+  T12^T12  hence 

T21 

v*  =»  v  ( 1  + - ) . 

T12 

We  have  chosen  ■  .00015  and  Tgj  *  .004,  from  which  we  obtain 
v*  «  .27666.  If  the  unit  of  time  Is  one  hour,  then  l/T^  »  9  months,  l/Tgj  ** 

10  days,  1/v*  «  3.5  hours. 

We  now  define  the  systems  III. A  and  III.B.  For  system  III. A,  we  have 
N  *  10,  R  *  1,  K  *  2,  Xj  *  Xg  *  .005,  *  tig  “  1*  vj  *  0,  v2  *  .27666,  q^  ■  0, 

q2  *  .2.  For  system  III.B,  there  Is  only  one  difference  :  u2  *  2,  the  service 
rate  Is  doubled  In  the  on-environment. 
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In  Table  III,  for  each  system,  the  first  column  represents  tne 
stationary  marginal  distribution  of  the  number  of  operating  machines;  the 
last  row  represents  the  stationary  marginal  probability  of  being  in  each 
environment;  in  the  remaining  two  columns  are  given  the  stationary  conditional 
distributions  of  the  number  of  operating  machines,  given  that  the  system  is  in 
the  corresponding  environment. 

The  comparison  of  the  column  III.A.O  in  Table  III  and  the  column  II. A 
In  Table  I  shows  that  the  two  distributions  are  very  similar,  but  the  marginal 
distribution  for  the  system  In  a  random  environment  has  a  longer  tail  towards 
the  states  with  few  operating  machines.  Me  also  observe  that  the  marginal 
distribution  is  not  a  good  descriptor  for  the  system  In  a  random  environment  : 
the  two  conditional  distributions  are  quite  different,  which  may  have  Important 
Implications,  even  If  there  Is  a  small  probability  of  being  in  the  on-environment. 

In  Table  IV  are  given,  for  the  same  two  systems,  the  values  of 
and  u1Q  ^(2),  for  0  <  k  <  9.  The  last  row  indicates  the  conditional  probability 
of  being  in  each  environment,  given  that  there  are  10  machines  In  operation. 

The  columns  III.A.l  and  III. A. 2  are  to  be  compared  to  the  column  II. A  in 
Table  II. 

Several  differences  may  be  observed.  Firstly,  the  time  g  to  com- 

3 

plete  failure  Is  shorter  by  a  factor  10  for  the  system  In  a  random  environment. 
Secondly,  the  values  of  2fjg  ^(1)  to  u^g  ^(1)  are  close  to  each  other  :  they 
differ  at  most  by  a  factor  2.  This  Is  due  to  the  Influence  of  the  on-environment 
It  takes  on  the  average  «*  6.6  10^  units  of  time  to  switch  from  environ¬ 

ment  1  to  environment  2,  at  which  time  the  connon-cause  failure  process  Is 
turned  on  and  brings  more  rapidly  the  system  towards  the  states  with  few  opera¬ 
ting  machines.  As  g(l)  Is  so  close  to  have  computed  ft10>6(l)  + 

Jfg  k(2),  for  0  <  k  <  5,  and  found  that  It  differs  from  the  corresponding  uw>k(l) 
by  at  most  70  units  of  time. 


We  conclude  that  the  systems  1 1.  A  and  1 1 1.  A  have  markedly  different 


dynamic  behaviours,  even  when  they  have  Identical  repair  facilities  and 
nearly  equal  stationary  failure  rates  : 

10 

^III.A  “  „ "  fpn,l  X1  +  pn,2  <x2  +  v2q2)]  *  -°666435, 

Is  close  to  A. 

When  examining  the  data  for  system  III .B,  we  observe  that  the  same 

comparisons  can  be  made  between  111 .B  and  II. A  as  between  III. A  and  II. A. 

10 

The  stationary  average  service  rate  for  system  III .B  Is  £  (p„  ,  p,  ♦  p.  ,  uo)  1 

n*0  n>1  1  c 

1.036145,  and  <pm  B  -  .0680697. 

The  remaining  examples  will  be  derived  from  the  system  III .B. 

We  shall  next  consider  systems  with  three  environments.  Environment  1 
Is  still  the  of f-envl ronment  where  no  common-cause  failure  may  occur.  Environ¬ 
ments  2  and  3  together  constitute  the  on-environment  where  common-cause  failure 
may  occur  at  a  constant  rate  v*,  but  with  different  failure  probabilities, 
which  we  denote  respectively  by  f2  and  f3,  In  order  to  distinguish  from  the 
probability  q2  In  the  preceding  model.  The  Infinitesimal  generator  T  for  envi¬ 
ronmental  changes  Is  chosen  as  follows  : 


12 

T12 

0 

21 

"(T21  +  T23^ 

T23 

21 

T32 

-<T21  +  T32> 

(6.1) 


thus  the  on -environment  has  exponential  duration  with  parameter  T21*  just  as 
for  the  systems  III. A  and  III. 8.  We  shall  set  T32  >  T23  and  f2  <  fj,  so  that 
the  on-periods  are  comprised  of  Intervals  of  relatively  low  failure  probability 
f2.  Interrupted  by  shorter  Intervals  of  higher  failure  probability  fj. 
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In  order  to  make  meaningful  comparisons  with  the  systems  I I. A  and 
III.B,  we  have  set,  as  before,  v*  »  v  (1  +  T21/T12),  so  that  the  expected 
time  between  successive  common -cause  failures  Is  the  same  for  all  systems. 

We  also  choose  f2  and  f3  such  that 

q2  -  a  f 2  +  (1-a)  f3, 

where  a  *  (^21  +  ^32^^21  +  ^32  +  ^23^  * 

and  Is  the  conditional  probability  of  being  In  environment  2,  given  that  the 
system  Is  either  In  environment  2  or  3.  This  choice  of  f2  and  f3  ensures  that, 
on  the  average,  the  probability  that  any  machine  falls  when  common-cause  fai¬ 
lure  occurs  Is  the  same  for  all  systems. 

The  system  III .C  Is  defined  as  follows  :  N  ■  10,  R  ■  1,  K  *  3,  Aj  ■  A2  ■ 
X3  ■  .005,  Uj  ■  1,  u2  ■  U3  ■  2,  vj  ■  0,  v2  *  Vj  *  .27666,  q^  ■  0,  f2  ■  .1504, 
f3  -  .36,  T12  ■  .00015,  T21  ■  .004,  T23  ■  .04,  T32  *  .125.  For  system  III .D. 
the  differences  are  as  follows  :  f2  »  .1801,  f3  ■  .7,  T32  «  1.  Thus,  we  have 
for  system  III .0  very  short  bursts  of  high  failure  probability. 

The  stationary  probability  distributions  are  respectively  given  In 
Tables  V  and  VI  :  In  column  0  the  marginal  distribution  of  the  number  of  ope¬ 
rating  machines;  In  columns  1  to  3  the  conditional  distributions,  given  the 
environment  state.  The  last  row  of  each  table  gives  the  marginal  distribution 
of  the  environment  state. 

We  compare  Tables  V  and  VI  with  the  columns  III.B  of  Table  III.  The 
sane  observation  that  were  made  earlier  are  valid  here  :  the  marginal  distribu¬ 
tion  has  a  greater  probability  mass  at  the  states  with  low  operating  machines. 

The  probabilities  corresponding  to  6  to  10  operating  machines  given  the  off- 
environment  (columns  III.B. 1,  III.C.l,  III.D.l)  are  nearly  equal  In  all  three 
systems.  The  conditional  distributions  given  the  on-environment  are  noticeably 
different  for  all  systems;  In  particular,  the  distribution  In  column  III.D.3  Is 
different  from  all  the  others,  In  that  It  is  more  or  less  uniform  for  n  between 
2  and  8. 
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In  Table  VII  are  given,  for  the  same  two  systems,  the  values  of 
“lO  »  f°r  *  <3,  0  <  k  <  9.  The  last  row  Indicates  the  conditional 

probability  of  being  in  each  environment,  given  that  there  are  10  machines 
In  operation.  We  observe  that  short  bursts  of  higher  failure  probability 
are  more  detrimental  to  the  performances  of  the  system,  since  the  values  of 
^10  *or  astern  III .C  are  nearly  consistantly  greater  than  those  for 

the  system  III.D,  the  exception  being  u^Q  fc(3)  for  high  values  of  k.  That 
exception  is  probably  due  to  the  very  short  duration  of  the  environment  3. 

Finally,  we  have  computed  that  ^  -  .0679624,  and  tpjjj  Q  =  .0680954. 

Our  general  conclusion  is  as  follows  :  if  one  considers  the  systems 
III. A  (or  III.B),  II. A  and  I. A  as  increasingly  simplified  approximations  of  the 
system  III.C  (or  the  system  III.D),  we  obtain  increasingly  optimistic  measures 
of  their  performance,  based  on  the  expected  first  passage  times  u10  k,  although 
the  systems  have  nearly  equal  stationary  expected  number  of  failures  per  uiit 
of  time. 

Type  IV.  For  our  last  numerical  examples,  we  shall  assume  that  the  server  does 
not  change  Instantaneously  the  service  rate  when  there  is  a  change  of  environ¬ 
ment.  More  precisely,  we  assume  that  the  environment  evolves  as  in  the  system 
III.C  and  III.D,  with  environmental  changes  governed  by  a  matrix  T  of  the  form 
(6.1).  The  "normal"  service  rate  in  the  off -environment  (environment  1)  is 
equal  to  yj.  When  the  system  enters  the  on-environment  (environment  states  2 
and  3),  the  server  notices  it  when  2  or  more  machines  fail  simultaneously,  or 
after  a  random  Interval  of  time  of  exponentially  distributed  duration,  with 
parameter  The  server  then  changes  the  service  rate  to  02*  The  service 
rate  remains  equal  to  yg  until  an  interval  of  time  has  elapsed  without  any 
failure,  that  Interval  of  time  is  exponentially  distributed,  with  parameter 

In  this  case,  we  need  to  describe  the  state  of  the  system  by  three 
Indices  (n.j.k),  where  n  denotes  the  number  of  machines  that  are  operating,  j 
denotes  the  environment  state,  and  k  «  1  or  2  to  indicate  the  service  rate  that 


Is  In  effect.  For  each  given  value  of  n.  Me  order  the  states  as  follows 
(n.1,1),  (n,2,l)v  (n,3,l) ,  (n»lt2),  (n,2,2)t  (nt3,2).  The  possible  tran~ 
sltlons  from  (n,j,k)  to  (n,j',k')  are  Indicated  below,  together  with  the 
corresponding  transition  rates. 


.  u2 


,  0  <  n  <  N-l 
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the  entries  represented  by  a  are  equal  to  zero,  the  entries  represented 
by  a  are  such  that  the  row-sims  of  Q  are  equal  to  zero. 

We  define  the  following  type  IV  system  :  N  «  10,  R  -  1,  K  *  3, 

Aj  »  *  .005,  yj  ■  1,  \»2  m  2,  Vj  ■  0,  V2  *  Vj  »  v*  *  .27666,  q^  »  0, 

f2  *  .1801,  fj  ■  .7,  ■  .00015,  Tgj  *  .004,  Tgj  *  .04,  =  1,  Cj=  t2  * 

v*/2  -  .13833. 

The  column  0  of  Table  VIII  gives  the  marginal  distribution  of  the 
number  of  operating  machines.  In  the  remaining  columns  are  given  the  condi¬ 
tional  distributions,  given  (j,lc)  where  j  Is  the  environment  state,  and  k  the 
Index  of  the  service  rate.  The  last  row  gives  the  marginal  stationary  proba¬ 
bilities  of  the  pairs  (J,k).  In  Table  IX  are  given  the  values  of  Ulo,n^’,c^, 
for  1  <  j  <  3,  k  *  1,2  ,  0  <  n  <9.  The  last  row  Indicates  the  conditional 
probabilities  of  being  In  the  state  (10,j,k),  given  that  there  are  10  operating 
machines. 

We  observe  from  the  last  row  of  Table  VIII  that  the  states  in  {(n,l,2), 

0  <  n  <  10}  are  essentially  states  of  transitions  towards  the  set  {(n,l,l), 

0  <  n  <  10}.  For  both  j  ■  2  and  3,  the  sets  {(n,j,l),  0  <  n  <  10}  and 
{(n,j,2),  0  <  n  <  10}  have  similar  probabilities,  but  different  distributions  : 
the  states  In  {(n.j.l),  0  <  n  <  10}  are  more  concentrated  than  those  In  { (n , j ,2) , 
0  <  n  <  10}  towards  large  values  of  n.  In  other  words,  towards  few  machines 
under  repair.  This  Is  seemingly  In  contradiction  with  the  inequality  uj  <  u2; 
this  Is  due  to  the  fact  that  transitions  from  (j,l)  to  (j,2)  occur 
whenever  there  are  multiple  failures. 

The  expected  first  passage  times  are  not  much  different  from  those  of 
the  system  III.D  (see  Table  VII). 
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Finally,  we  mention  that  the  program  Implementing  Algorithm  E 
has  been  written  In  FORTRAN  77  and  run  on  a  CDC  170/750.  Care  has  been 
taken  not  to  waste  memory  space,  by  making  sure  that  no  space  Is  reserved 
for  matrices  IT,  .  with  j  >  1,  Q,  .  with  j  >  n+1,  or  vectors  u,  .  with  j  >  i. 

I  »J  1  »J  1  »J 

In  other  respects,  the  Implementation  was  straight  forward.  For  systems  with 
N  -  10,  the  program  uses  on  the  average  .2  seconds  of  central  processor  time 
for  K  «  3,  and  .6  seconds  for  K  =  6. 
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.933342 

.950059 

.487564 

.941804 

.950174 

.718606 

Env. 

.963855 

.036145 

.963855 

.036145 

Table  III 

Stationary  distributions  for  systems  III. A  and  III.B. 
Missing  numbers  are  less  than  10~6. 
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III. A. 2 
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III.B. 2 
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2  106 
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16  10 3 
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CO 

o 

«— 4 

00 
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7  103 

443.9 

7  103 
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4  103 

163.7 

4  103 
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454.6 

15.1 

454.7 

15.4 

20.0 

3.6 

20.0 

3.6 

.981118 

.018882 

.972421 

.027579 

Table  IV 

Expected  time,  starting  from  (10, j),  until  there 
remain  k  or  less  operating  machines,  for  systems 
III. A  and  III.B. 
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Table  V 


Stationary  distributions  for  system  III.C. 
Missing  numbers  are  less  than  10‘6. 
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Table  VI 


Stationary  distributions  for  system  III.D 
Missing  numbers  are  less  than  10'®. 
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Table  VII 


Expected  time,  starting  from  (10, j),  until  there  remain 
k  or  less  operating  machines,  for  systems  III.C  and  III 
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Table  VIII 


Stationary  distribution  for  a  Type  IV  system. 
Missing  numbers  are  less  than  10"®. 
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(1 
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8 

4 

103 
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7 

163 

3 

8 
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7 
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3 
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0 

457 

7 

16 
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15 
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0 

3. 

7 

3. 

6 

20 

0 

3 

7 

3 
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.974497 

.010184 

.000369 

.000578 

.013876 

.000496 

Table  IX 


Expected  time,  starting  from  (10,1, k),  until  there 
remain  n  or  less  operating  machines,  for  a  type  IV 
system. 
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APPENDIX  A 


First  order  derivatives  of  the  matrices  Dn(5) 

The  equations  (3.13)  and  (3.14)  may  be  written  as 

VE><  EI  -  «o,o>  ■  '• 

°n<'»  El  V«n,r  *  £  Vk  <*.„<'»  ”  *<■««-»• 


By  differentiating  both  sides  of  these  equations  with  respect  to  €,  we 
obtain  that 


DqU)(  tl  -  Q0t0)  +  D0(O  -  0, 

°n<5><  -  Vn  ’  £  Vk  +  Dn<*>  <X  “  £  Vic  6k.n«>)  *  0. 


k-0 
1  <n  <N-1, 


or  0^(0  -  -  (D0(O)Z  . 


n-1 


VO  -  -  Dn<e)(I  -  £  Qn,k  G^n(0)  Dn(0.  for  1  <1.0-1. 


(A.  1) 

(A.2) 


First  order  derivatives  of  the  matrices  Bn(() 

The  equations  (4.5)  and  (4.6)  may  be  written  as 

tyOUI  -  0*,*)  •  I. 

•  V.  -  "n.iHl  W'»  ■  '• 

Ip  differentiating  both  sides  of  these  equations  with  respect  to  c,  we 

ehteln 

*  (^ivi<>  4  V<)  *  °* 


A. 2 


or  6jJU)  ■  -  (0N(e))z.  (A.3) 

ana  5;(«(M  -  Q„,„  -  0„>lW  6^, ,„(£))  *  6„<5)<I  -  WE»  ■ 


or 


";(£)  ■  -  B„(£)(I  -  6^1,„(S))  »„<«>.  for  1  <■  <«-!•  (*-4) 


B.l 


APPENDIX  B 

Algorithm  C.  Evaluation  of  the  stationary  probability  vector  £  ■  (j>Q, 
and  the  expected  first  passage  times  n,  ,  for  all  pairs  of  levels  n 

Cl  (Initial  computations  :  the  matrices  rrn  k,  and  the  vectors  uf|}. 

CN  QN,N  ; 

determine  u^  from  (4.12); 

for  k  from  0  to  N-l  do  determine  nN  ^  from  (2.4) ; 

for  n  from  N-l  to  1  step  -1  do 

begin  determine  Cn  from  (2.3); 
determine  un  from  (4.13); 

for  k  from  0  to  n-l  do  determine  nn  k  from  (2.5) 

end; 

determine  Cq  from  (2.3); 

C2  (Determination  of  the  vectors  and  n,) 

i^j  :■  solution  of  the  system  2q  Cq  *  0,  Hq  e  ■  1; 

C0  Q0,0; 

M.1,0  “l; 

for  n  from  1  to  N  do  determine  Q  from  (4.16) ; 
determine  Uq  ^  from  (3.17); 
determine  rQ  ^  from  (3.4); 
for  m  from  1  to  N-l  do 


•  *  •  »£fl) » 

♦  n‘. 


B.2 


! 

i 

i 


begin  determine  it  from  (2.7); 

m 

re-normal  ize  the  vectors  itrt  to  n_  so  that  I  e  •  1; 
-  -0  -m  1s0  -i  - 

for  n  from  iih-1  to  N  do  determine  u _ from  (4.16); 

-  -  — - — n  ,m  * 

determine  Cn  from  (3.3); 
determine  ^  from  (3.18) ; 
determine  rm  from  (3.4); 
for  rr  from  0  to  m-1  do 
begin  determine  un  ^  from  (3.19); 

determine  rn  ^  from  (3.5) 
end 


end; 

determine  tt^  from  (2.7); 
renormalize  the  .„-ctors  iig 


N 

to  it*  so  that  I  n  e  *  1. 
-H  n«0  ~n  ~ 


At  the  end  of  the  algorithm,  the  vectors  are  equal  to  the  vectors 
0  <  n  <  N.  We  systematically  renormalize  the  vectors  In  order  to  avoid 
overflow  problems. 


Algorithm  D.  Recursive  evaluation  of  the  vectors  v^,  defined  by  v^  «  uN  k  , 
0  <k  <N-1. 

01  (Initial  computations) 

CN  QN,N; 

determine  u^  from  (4.12); 

•’*  (equation  (4.17)) 


B.3 


02  (Main  loop,  at  the  k  th  step,  is  determined) 
for  k  from  N-l  to  1  step  -1  do 

begin  for  n  from  N  to  k+1  step  -1  do  determine  nR  k  from  (2.4)  and  (2.5); 
determine  from  (2.3); 

determine  from  (4.13); 

Ufc  u,;  (equation  (4.17)) 

for  n  from  k+1  to  N  do  determine  un  from  (4.18); 

\-l  HN,k-l 

end. 

Algorithm  E.  Evaluation  of  the  stationary  probability  vector  £  =  (Pg»- • • .fy) » 
and  the  expected  first  passage  times  ul  ,,,  for  0  <  k  <  n  <  N.  This  algorithm 

*TI 

combines  Algorithm  D  and  part  of  Algorithm  C. 

El  (Initial  computations) 

CN  :*■  Qn>n  ; 

determine  u^  from  (4.12); 

for  k  from  0  to  N-l  do  determine  n.^  ^  from  (2.4) ; 

E2  (Determination  of  the  vectors  u^  k  and  preparation  to  the  evaluation  of  the 
vector  £) 

U||  :■  u,  ;  (equation  (4.17)) 

for  n  from  N-l  to  1  step  -1  do 

begin  determine  Cp  from  (2.3); 

determine  uR  from  (4.13); 

Vn-1  ;  (equation  (4.17)) 

for  m  from  n+1  to  N  do  determine  if _ ,  from  (4.18); 

—  -  —  —  —  -  -m,n-i 

for  k  from  0  to  n-l  do  determine  n  k  from  (2.5) 
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Infinitesimal  generator  for  Type  III. 

We  assume  N  =  4,  R  *  2  and  K  =  3.  Entries  represented  by  a 
are  equal  to  zero,  entries  represented  by  a  are  equal  to  minus  the  sum 
of  the  off-diagonal  entries  on  the  same  row. 
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